Method and system for automating a user interface

ABSTRACT

Certain embodiments relate to providing automated, custom displays of images and tools for manipulating the images. Certain embodiments include a method for providing an automated user interface for healthcare workers. Certain embodiments of the method employ a network to execute the steps of retrieving an image from an image archive, processing the image to generate image-specific information, identifying data related to the image, mining the data related to the image, and displaying the image along with a custom selection of user interface tools.

RELATED APPLICATIONS

[Not Applicable]

FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

[Not Applicable]

[MICROFICHE/COPYRIGHT REFERENCE]

[Not Applicable]

BACKGROUND OF THE INVENTION

Embodiments of the present method and system relate generally toelectronic data collection and display in a healthcare setting.Particularly, certain embodiments relate to providing automated, customdisplays of images and tools for manipulating the images.

Healthcare facilities often employ certain types of digital diagnosticimaging modalities, such as computed tomography, magnetic resonanceimaging, ultrasound imaging, and X-ray imaging. As part of thetherapeutic or diagnostic process, healthcare workers spend aconsiderable amount of time evaluating and interpreting images, as wellas preparing clinical reports. The clinical reports often contain directreferences to images as well as data gathered from the images.

To facilitate the evaluation and interpretation of images,computer-based systems have been developed to present the data to auser, typically a radiologist but possibly any of a number of otherusers in a healthcare setting. As with most computer-based systems, theuser interacts with the system via input devices such as a mouse and/ora keyboard. Users who spend considerable time with such devices maydevelop conditions, such as carpal tunnel syndrome, common tocomputer-intensive workplaces. This is in part due to the reliance oninput devices such as a mouse and/or a keyboard to perform what areoften repetitive tasks.

Moreover, some of the analysis performed on images by users ispredictable. Past images and past clinical reports related to a givenpatient may contain information that is helpful in determining what partor parts of a current image may be of interest to a user. In this way,certain parts of image analysis may be amenable to automation. Areliable way of automating image analysis based on past clinical dataand past images could enable users to interpret and evaluate images moreefficiently.

Thus, there is a need for a system and method to reduce the reliance onuser input devices to perform the repetitive evaluation and analysis ofimages. There is a further need for a means of automating thepredictable aspects of image interpretation and analysis.

BRIEF SUMMARY OF THE INVENTION

Certain embodiments of the present invention include a method forproviding an automated user interface for healthcare workers interactingwith an image analysis system. Certain embodiments of the method includethe steps of retrieving an image from an image archive, processing theimage to generate image-specific information, identifying data relatedto the image, mining the data related to the image, and displaying theimage along with a custom selection of user interface tools.

Certain embodiments of the present invention include an automated userinterface for use in a healthcare setting. According to certainembodiments, the automated user interface comprises a network, an imagearchive connected to the network, a data archive connected to thenetwork, and a workstation connected to the network. According tocertain embodiments, the workstation has a user interface device, imagedisplay capabilities, and user interface tools. According to certainembodiments, a rules engine is connected to the network, wherein therules engine processes an archived image for display, identifies andmines archived data related to the archived image, and displays theimage via the workstation along with a custom selection of userinterface tools.

BRIEF DESCRIPTION OF SEVERAL VIEWS OF THE DRAWINGS

FIG. 1 illustrates a flow diagram for a method for automaticallyproviding a custom user interface along with an image or images inaccordance with an embodiment of the present invention.

FIG. 2 illustrates a workstation display employing a customized userinterface in accordance with an example of one embodiment of the presentinvention.

The foregoing summary, as well as the following detailed description ofcertain embodiments of the present invention, will be better understoodwhen read in conjunction with the appended drawings. For the purpose ofillustrating the invention, certain embodiments are shown in thedrawings. It should be understood, however, that the present inventionis not limited to the arrangements and instrumentalities shown in theattached drawings.

DETAILED DESCRIPTION OF THE INVENTION

The methods and systems of certain embodiments of the present inventioncontemplate that a user, typically a radiologist but potentially anyhealthcare worker or clinician, has an interest in viewing and analyzingimages and data collected during diagnostic or therapeutic treatment ofa subject. To this end, the user interfaces with a computerized imageand data display system, typically through the use of a workstation. Theworkstation may be a standalone system or it may be a networked system.In the case of a networked system, the network may extend within asingle healthcare facility or it may extend across multiple healthcarefacilities. The networked system may be integrated with other networkedsystems, including networks carrying data other than images, forexample, a network carrying patient demographic data. In any case, theuser has access to stored images and stored data at least via theworkstation.

Thus, in certain embodiments of methods and systems of the presentinvention the user interacts with a workstation to view and analyzespecific images or series of images to further the diagnosis ortreatment of a subject. Computerized image analysis and imagemanipulation tools facilitate image analysis by the user on theworkstation. Such tools form part of the user interface of theworkstation. Certain embodiment of the methods and systems of thepresent invention provide an automated and customizable user interface.

FIG. 1 illustrates a flow diagram for a method 100 for automaticallyproviding a custom user interface along with an image or images inaccordance with an embodiment of the present invention. In retrievalstep 110 of method 100, an image or images are retrieved from an imagearchive. Typically, retrieval step 110 takes place in response to auser's request to a view an image associated with a specific patient.Retrieval step 110 may involve retrieving a single image, or it mayinvolve retrieving a series of images.

In the case of a series of images, the series may be images collectedduring a single patient imaging event, such as a series of “imageslices” captured during a computerized-tomography imaging session. Ofcourse, other imaging modalities capable of producing a series of imagesor having their images collected in a series are also contemplated bythis embodiment. In addition, the series of images may be imagescollected during different imaging events. The images may have beenpreviously associated with each other by a clinician to form a series.Or, the images may be currently associated with each other to form aseries by the clinician during retrieval step 110. For example, therequest data input from the user may contain multiple image requests,thus potentially associated the images.

The image archive from which the image is retrieved in step 110 may bean archive local to the workstation or it may be from a networked dataarchive. The workstation may be part of a Picture Archive andCommunication System (PACS).

Referring again to FIG. 1, in image processing step 120 an image isprocessed to gather information relevant or helpful to the method of oneembodiment of the present invention. Again, the image may be a singleimage or it may be a series of images. Image processing step 120 mayinclude identifying the similarities and/or differences among featurespresent in each image of a series of images. For example, in a series of“images slices” which recreate a three-dimensional section of a subject,differences among the images can be used to identify irregularities inthe tissue being imaged. Similarly, images or series of images collectedduring different imaging sessions for a certain subject can be processedto identify similarities or differences among the images.

In any case, one object of image processing step 120 in one embodimentof the present invention is to identify image information of potentialinterest to the user. For example, in the case of a series of “imagesslices” which recreate a three-dimensional section of a subject, thepresence of systematic differences in images features may indicate thepresence of abnormal tissue growth. The presence of this abnormal growthis potentially of interest to the user and immediately viewing theportion of the image containing the abnormal growth can expedite adiagnosis of the subject's condition. Thus, image processing step 120 isuseful at least to automate the collection of image-specific informationto be displayed to the user as part of the customized user interface.

Moreover, while the above discussion highlights a processing methodologythat seeks to compare images to find differences or similarities inimage features, other image processing methodologies may be suitable foridentifying information of potential interest to the user. Such othermethodologies may include, for example, performing mathematical analysisof the images to generate information not readily apparent upon visualinspection of the image.

Another object of image processing step 120 in one embodiment of thepresent invention is to help customize the selection of tools presentedto the user as part of the user interface. For example, in theabove-mentioned case of the presence of systematic difference in imagesfeatures that may indicate the presence of abnormal tissue growth, auser is likely to want to measure the growth as part of the diagnosisprocess. Immediately presenting the measurement tool as part of thecustom user interface can expedite a diagnosis. Thus, image processingstep 120 is useful at least to customize the selection of user interfacetools to be displayed to the user as part of the customized userinterface.

Image processing step 120 may take place as part of a software routineavailable locally on the workstation. Alternatively, the imageprocessing routine may be part of a software routine available on theimage archive or on a network linking the archive and the workstation.

Referring again to FIG. 1, during data identifying step 130, data thatis relevant or potentially relevant to a users diagnosis of a subjectscondition is identified according to one embodiment of the presentinvention. In order to identify the relevant data, identifying step 130may be aided through the use of data input. For example, patientdemographic information may be helpful for data identifying step 130 tolocate clinical reports or other data kept in a data archive andrelevant to the diagnosis.

In the example where patient demographic information is used, suchinformation may be provided directly by the user at the same time theuser requests to a view an image as part of image retrieval step 110. Insuch a case, the user may directly request the retrieval of a specificclinical report or other data or sets of data stored on a data archiveby inputting patient demographic data or other patient identifying data.With such a direct request from the user, patient demographicinformation may readily be extracted from the direct request andsupplied as input to data identifying step 130. Data identifying step130 may then use the patient demographic information input to identifyother patient data potentially relevant for automatically providing acustomized user interface. Examples of potentially relevant data includeother clinical reports for the subject available in a data archive butnot specifically requested by the user. Further examples of potentiallyrelevant data include patient specific data stored on other dataarchives networked with the workstation. Data identifying step 130 mayoptionally also perform the task of retrieving the specific datarequested by the user.

In other examples related to providing input to data identifying step130 according to one embodiment of the present invention, the user mayindirectly provide input useful for identifying related data. Forexample, a user may simply request an image or series of images but notalso request the retrieval of a related clinical report. In such case,data identifying step 130 may evaluate data or meta-data associated withthe requested image to provide the input potentially necessary toidentify data related to the requested image. As described above, manycurrent imaging system use data objects associated with images in theDICOM format. Thus, data identifying step 130 may evaluate the DICOMdata objects associated with a requested image and use certain dataobjects as keys for further data identification. DICOM data objects mayinclude, for example, patient demographic information. Once such patientdemographic information is obtained, data identifying step 130 canproceed as outlined above to locate potentially relevant data useful forautomatically customizing a user interface. Additionally, images may beassociated with each other as a result of data identifying step 130,according to one embodiment of the present invention. For example, theDICOM data objects may include information relevant for identifyingother images in the image archive that can be processed according toimage processing step 120 to yield further opportunities to customizingor refining the customization of the user interface.

Referring again to FIG. 1, in data mining step 140, data identifiedthrough data identifying step 130 is evaluated to facilitate theautomated customization of a user interface, according to one embodimentof the present invention. In such an embodiment, data mining step 140involves scanning the identified data for text or other data that arepotentially relevant for customizing the user interface. Text and datacan be identified as relevant following at least a few differentprocesses.

According to one embodiment of the present invention, the data mining ofdata mining step 140 may occur with a set of pre-determined rules.Pre-determined rules relate to patterns of data known to occur or knownto be likely to occur in a given data set. Such data patterns mayinvolve small or large pieces of an individual data set. The datapatterns may also occur across individual data sets, such as a series ofclinical reports.

One example of a pre-determined rule involves a data pattern having anumerical entry followed by text typically known to connote a unit ofmeasure, such as “14 mm,” which could be referred to as a measurementrule. The presence of such a measurement rule may indicate, for example,a previous identification and diagnosis of a tissue abnormality. Thepresence of the measurement rule may then trigger a search in the nearbydata for a second level of pre-determined rules common to the diagnosisof a tissue abnormality, such as for example the text string “nodule.”Many other second level pre-determined rules are possible and are withinthe scope of data mining step 140. Further, data mining step 140 mayinvolve nesting the pre-determined rules described above in a differentorder such that data mining step 140 searches first for the text string“nodule” and the measurement rule is a second tier rule. Of course, thenesting of rules is not limited to two levels and can have many levels.Moreover, searches for different pre-determined rules may occur inparallel in the same data set and may also occur recursively in a givendata set.

According to one embodiment of the present invention, data mining step140 may generate learned rules. Learned rules involve data patterns thatemerge from a data set as that data set is evaluated. As in the case ofpre-determined rules, learned rules may emerge for large or smallpatterns of data and may extend across multiple data sets. Importantdata patterns are desirable for use as learned rules to facilitateexecution of data mining step 140. The importance of a data pattern isdetermined at least in part by several factors, including the datapattern's frequency, proximity to other key data, and the relevance ofthe data contained within the pattern.

Frequently occurring data patterns may be important in a given data setas a means of determining the identity of data that has been of interestto a user. A frequent text string such as “fibrous mass” may indicatethat such tissue abnormality has been the subject of past user interest.Conversely, a single occurrence of the text string “fibrous mass” in alarge data set may indicate that such tissue abnormality has beenidentified in the past and is not of interest. As with the above exampleof a second level of search using predetermined rule after locating afirst pre-determined rule, a second level of search, and multiplefurther levels of search, can be used to refine the importance of anemerging data pattern.

In one embodiment of the present invention, data mining step 140 may useonly pre-determined rules, or it may use only learned rules. Preferably,data mining step 140 uses a combination of pre-determined rules andlearned rules to locate potentially relevant data useful forautomatically customizing a user interface. Further, a learned rule maybe added to the set of pre-determined rules available for data mining.The added learned rule may be added in a specific form, such as forexample the text string “fibrous mass,” or it may be added in a moregeneric form aids in identifying patterns that deviate somewhat from thespecific learned rule.

In one embodiment of the present invention, pre-determined rules used indata mining step 140 may come from data or meta-data evaluated duringdata identifying step 130. For example, data identifying step 130 mayevaluate the DICOM data objects associated with a requested image anduse certain data objects as output for the formation of a pre-determinedrule useful in data mining step 140. Thus, data identifying step 140 mayprovide content related to data mining step 140 in addition to locatingrelevant data.

Referring again to FIG. 1, in customizing step 150 the results of imageprocessing step 120, data identifying step 130, and data mining step 140may be used to customize the user interface, according to one embodimentof the present invention. In one embodiment, as custom user interface isone in which area or areas of interest in an image are highlighted forthe user and useful user interface tools are presented to the user.

For example, as a result of image processing step 120, certain featuressuch as tissue abnormalities may have been identified in an image orseries of images. In one embodiment, customizing step 150 correlates thefeature size and location on the image or series of images and selectsan appropriate image size, image alignment, and/or image magnificationto highlight the tissue abnormality for the user. In the event thatimage processing step 120 has identified multiple image features in animage or series of images, customizing step 150 may apply rulesregarding the priority of display of such multiple features.

For example, if image processing step 120 has identified two features ofinterest to a user, such as a fibrous mass and a potential fluid sac,customizing step 150 may apply rules that indicate a priority for thepotential fluid sac. Thus, customizing step 150 may select imageorientation and magnification parameters that first highlight the fluidsac. Customizing step 150 may further select image orientation andmagnification parameters that highlight the fibrous mass for laterdisplay to the user.

In one embodiment of the present invention in which customizing step 150determines the priority of display for image features, key dataidentified during data mining step 140 may also be used to prioritizeimage features. For example, if data mining step 140 generatesinformation from clinical reports that the size and shape of a certainimage feature has been frequently measured, then that tissue feature mayreceive a high display priority since it appears to be of interest to auser. Customizing step 150 may also have rules to reconcile conflictingpriorities. For example, when an otherwise low priority image featurehas been frequently measured in past clinical reports, customizing step150 may raise the display priority of such otherwise low priority imagefeature.

In one embodiment of the present invention, customizing step 150 maygenerate a list of user interface tools to be presented to a user alongwith the customized image orientation and magnification. For example, ifimage processing step 120 and/or data mining step 140 has identified animage feature of interest for measurement by a user, customizing step150 may assign a high priority to the measurement tool. Further, in thisexample customizing step 150 may also assign a high priority to anannotation tool. Thus, customizing step 150 is useful for automaticallycustomizing the user interface in that it may provide immediate accessto interesting features in an image, displayed in a useful orientationand magnification, and may also provide immediate access to specificimage manipulation tools.

Referring again to FIG. 1, in displaying step 160 the image or imagesretrieved from the archive may be displayed for the user following thecustom parameters generated by customizing step 160, according to oneembodiment of the present invention. Additionally, a custom selection oftools may be displayed for the user in displaying step 160. Optionally,data sets identified through data identifying step 130 may also bedisplayed for the user. Further, certain data strings may be highlightedor featured for the user as part of displaying step 160.

The steps described above are illustrated in FIG. 1 as occurringsequentially. However, in certain embodiments of the present invention,some or all of the steps described above may occur in parallel. Further,some of the steps described above may be collapsed into a single stepaccording to certain embodiments of the present invention. Of course,modifications in the timing, order, or number of the steps of the methodof the present invention are contemplated and are within the scope ofcertain embodiments of the method. Further, the steps of the method maybe carried out repeatedly in a loop according to certain embodiments ofthe present invention. The method steps described above lay out aflexible architecture for achieving an automated, customizable userinterface.

Within the steps of certain embodiments of the method of the presentinvention, certain rules or sets of rules may be necessary to carry outcertain functions. For example, in one embodiment of the method of thepresent invention, data mining step 140 employs certain rules thatidentify data patterns in order to carry out the mining function.Similarly, according to one embodiment, customizing step 150 employscertain priority rules to carry out the customizing function. Such anassembly of rules may be referred to as a rules engine. A rules enginemay contain rules unique to a single method step or multiple methodsteps, according to certain embodiments of the present invention. Arules engine may also contain rules common to multiple method steps, andmay contain a nested set of rules. Rules may be applied to data inroutines or subroutines.

FIG. 2 illustrates a workstation display 200 employing a customized userinterface in accordance with an example of one embodiment of the presentinvention. Workstation display 200 includes image 210, measurement tool220, and annotation box 230, according to one embodiment of the presentinvention. FIG. 2 illustrates an example of a customized user interface,in that a specific image is displayed along with certain user interfacetools. In this example, the selected user interface tools aremeasurement tool 220 and annotation box 230 as a result of the imageprocessing and data mining of one embodiment of the present invention.Although only two user interface tools are depicted in FIG. 2, andnumber of user interface tools may be selected for immediate orsequential display. Example of user interface tools include panning,selecting a region of interest, annotation, and window leveling.

The methods of embodiments of the present invention may be executedusing systems including a workstation, an image archive, and a dataarchive. In one embodiment, the workstation may be a standalone systemwith images and data archived on the workstation. Preferably, theworkstation, image archive, and data archive are networked together. Thenetwork may include a plurality of workstations as well as multipleimage archive and data archives, according to embodiments of the presentinvention.

According to certain embodiments, the workstation has a user interfacedevice, image display capabilities, and user interface tools. Accordingto certain embodiments, a rules engine is connected to the network,wherein the rules engine processes an archived image for display,identifies and mines archived data related to the archived image, anddisplays the image via the workstation along with a custom selection ofuser interface tools.

The technical effects of certain embodiments of the present method andsystem are to retrieve an image from an image archive, to process theimage to generate image-specific information, to identify data relatedto the image, to mine the data related to the image, and to display theimage along with a custom selection of user interface tools.

As described above, the methods of embodiments of the present inventionmay employ routines and subroutines to carry out certain method steps.The routines and subroutines may be stored on computer readable mediaaccessible to a workstation according to one embodiment. Accessibilityincludes without limitation having the media stored within a workstationor stored on a network connected to a workstation. The routines andsubroutines may be used to implement the rules or rules enginesdescribed above, in accordance with an embodiment of the presentinvention.

EXAMPLE

A radiologist in an academic/teaching hospital specializes in CTreadings. So, the radiologist's workflow is normally going through along worklist with CT exams. The radiologist's focus is both on qualityand productivity. Under prior art conditions, the radiologist opensevery exam, views the images, historical images, historical reports andthen does image manipulation like zoom, windowlevel, pan to get theright quality of the images and then performs annotations, enters keyimage notes and then creates reports. This whole prior art process takesmany minutes, repeating the process for every exam.

Using an embodiment of the present invention for patient X, theradiologist is assisted by providing some of patient X's informationahead of time. When the radiologist opens an exam—the image manipulationis done and needs just minor work. Regions of interest are marked andannotations are entered based on how he has been doing it, that is, thesystem learns based on past practice. So, it reduces the time it takesfor the radiologist to read every exam.

One potential advantage of the system and method of the presentinvention is a reduced reliance on mouse and keyboard. Since the user ispresented with a customized user interface tailored to the specificimage and data displayed on the workstation, there is less of a need forthe user to employ the mouse and keyboard to select appropriate usertools. This reduced reliance on a mouse and keyboard may in turn yieldfurther advantages. For example, less reliance on a mouse and keyboardmay lessen the incidence of carpal tunnel syndrome in users of thesystem and method of the present invention. Another potential advantageof the system and method of the present invention is that of focusingthe limited time of a busy clinician on medically-oriented tasks ratherthan software-oriented tasks. This increased medical focus potentiallyprovides a benefit to the user in that the user's time is occupied moreefficiently. Moreover, it potentially provides a benefit to the subjectin that the user is able to turn more attention to the interpretationand analysis of diagnostic data rather than spending time navigatingsoftware.

While the invention has been described with reference to certainembodiments, it will be understood by those skilled in the art thatvarious changes may be made and equivalents may be substituted withoutdeparting from the scope of the invention. In addition, manymodifications may be made to adapt a particular situation or material tothe teachings of the invention without departing from its scope.Therefore, it is intended that the invention not be limited to theparticular embodiment disclosed, but that the invention will include allembodiments falling within the scope of the appended claims.

1. An automated user interface for use in a healthcare settingcomprising: a network; an image archive connected to the network; a dataarchive connected to the network; a workstation connected to thenetwork, the workstation having a user interface device, image displaycapabilities, and user interface tools; and a rules engine connected tothe network, wherein the rules engine processes an archived image fordisplay, identifies and mines archived data related to the archivedimage, and displays the image via the workstation along with a customselection of user interface tools.
 2. The user interface of claim 1wherein the rules engine determines the custom selection of userinterface tools based on information gathered during the processing ofthe archived image.
 3. The user interface of claim 1 wherein the rulesengine determines the custom selection of user interface tools based oncomparison of information gathered during the processing of the archivedimage to information gathered during the mining of the archived data. 4.The user interface of claim 1 wherein the rules engine mines thearchived data based on information gathered during the processing of thearchived image.
 5. The user interface of claim 1 wherein the customselection of user interface tools is selected from the group consistingof annotation boxes, zoom levels, measurement tools, panning, selectinga region of interest, and window leveling.
 6. The user interface ofclaim 1 wherein the image archive is a Picture Archive and CommunicationSystem.
 7. The user interface of claim 1 wherein the data archive is aRadiology Information System (RIS), a Hospital Information System (HIS),or a Clinical Information System (CIS).
 8. A method for providing anautomated user interface for healthcare workers using a networked dataarchive and image archive comprising the steps of: retrieving an imagefrom the image archive; processing the image to generate image-specificinformation; identifying data related to the image; mining the data;displaying the image and a custom selection of user interface tools. 9.The method of claim 8 wherein the data is identified based on input froma user.
 10. The method of claim 8 wherein the data is identified basedon the image-specific information.
 11. The method of claim 8 wherein thedata is mined based on input from a user.
 12. The method of claim 8wherein the data is mined based on the image-specific information. 13.The method of claim 8 wherein the custom selection of user interfacetools is based on input from a user.
 14. The method of claim 8 whereinthe custom selection of user interface tools is based on theimage-specific information.
 15. The method of claim 8 wherein the imagearchive is a Picture Archive and Communication System.
 16. The method ofclaim 8 wherein the data archive is a Radiology Information System(RIS), a Hospital Information System (HIS), or a Clinical InformationSystem (CIS).
 17. A computer readable storage medium including a set ofinstructions for a computer, the set of instructions comprising: aretrieval routine for retrieving images from an image archive; aprocessing routine for processing images to gather image-specificinformation; a mining routine for mining data; a displaying routine fordisplaying images; and an interface routine for providing a customselection of user interface tools.
 18. The computer readable medium ofclaim 17, wherein the set of instructions further comprises a miningrules routine for applying the image specific data to the mining routineto generate mining topics based on the image specific data.
 19. Thecomputer readable medium of claim 17, wherein the set of instructionsfurther comprises a display rules routine for applying the imagespecific data to the displaying routine to providing the custom imagedisplay based on the image specific data.
 20. The computer readablemedium of claim 17, wherein the set of instructions further comprises aninterface rules routine for applying the image specific data to theinterface routine to providing the custom selection of user interfacetools based on the image specific data.